The present invention relates generally to processor systems, and, more particularly, to Non-Uniform-Memory-Architecture (NUMA) systems, having a plurality of nodes which each include a processor and a portion of the total memory of the NUMA system.
NUMA systems are a developing area of processor systems which permit sharing a total main memory among a plurality of nodes of the system. The overhead of accessing pages of the main memory depends on the location of the pages since access to pages in the same node as the processor is faster than access to memory located at another node. Therefore, in NUMA systems, performance can improve if the operating system migrates or replicates specific pages to other nodes since this reduces the coherence bandwidth requirements and the average latency of the memory accesses. In migration, the single copy of the page is transferred to another node, whereas, in replication, an extra copy of the page is provided at another node.
In order for the operating system to implement replication and migration, the best candidate pages for migration or replication need to be identified. One of the early systems directed to identifying good candidates was U.S. Pat No. 5,269,013 to Abramson et al. The Abramson et al. scheme uses a hardware buffer structure on each node to store samples from the stream of memory requests. Periodically, an operating system daemon examines the contents of the buffer memory. The sampled information from this is used to maintain an ordered list of pages in local memory order where the most frequently referenced page will be at the top of the list. The sample information is also used to maintain a hash table of remote pages accessed by the node processor. The daemon also maintains a counter that represents the average number of samples that refer to local pages. If the number of accesses to a remote page exceeds the average number of samples referring to local pages, the remote page is considered a candidate for migration. If no free pages exist in the local node, the least recently referenced page will be moved to a remote memory.
U.S. Pat. No. 5,727,150 to Laudon et al. shows another system for migrating or replicating a page. In the approach disclosed there, every page in the system is associated with a set of counters that count the read/write requests to a page on a per node basis. For each page, the operating system can also define a threshold value stored along with the counters. If the difference between the number of requests from the local node and a remote node exceeds the threshold for the page, an interrupt is delivered to the node processor in order to notify the operating system of this event. The operating system can then choose to migrate or replicate the page.
Although the above described systems use page migration/replication policies that are capable of identifying the best candidate pages for migration/replication and then performing these operations, they suffer from a number of drawbacks. For example, in these systems, information is collected for all pages in the system. By virtue of this, the storage requirements are very large.
In addition, in systems such as described in the Laudon et al. patent, an interrupt is triggered when a page is declared a possible candidate for migration/replication. This incurs an undesirable interrupt overhead. In systems such as that disclosed in the Abramson et al. patent, an operating system daemon process is required to maintain the counters, thus incurring an undesirable software overhead in the operating system. Also, in the Abramson et al. arrangement, if the operating system is over-committed and does not examine the samples, the maintenance of the counters will cease. Accordingly, it is necessary to design the operating system to avoid this problem.
A processor system, and method of operating the same, is provided which includes a plurality of nodes, each including a processor and a portion of the total main memory of the processor system. A counter is provided which tracks certain ones of a type of event which occur in the processor system, determined to be the most interesting ones in accordance with a predetermined standard. On the other hand, the counter discards other ones of the same type of event determined by the standard to be less interesting.